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Abstract 

This  paper  presents  a  technique  to  simulate  and  evaluate  a  system  once  the  system  scenarios 
are  available  without  any  simulation  programming.  This  is  different  from  traditional  simulation 
where  simulation  code  and  the  system  specification  are  separately  developed  by  human  engineer 
and  potential  gaps  between  them  might  be  introduced.  Another  significant  advantage  of  this 
approach  is  that  the  scenarios  specified  do  not  need  to  be  complete  or  consistent.  Inconsistency 
and  incompleteness,  as  well  as  safety,  performance,  and  behavior  problems,  can  be  detected  by 
the  simulation  via  various  dynamic  analyses.  This  technique  is  a  part  of  Scenario-Driven  System 
Engineering  (SDSE)  that  is  being  developed  for  Command-and-Control  systems. 

Keywords:  Scenarios,  ACDATE,  Simulation,  Scenario-Driven  System  Engineering, 
Completeness  and  Consistency  Analysis,  Safety  Analysis. 

1.  Introduction 

Euture  Command  and  Control  (C2)  systems  need  to  operate  within  an  integrated  grid-based 
network-centric  environment  that  allows  rapid  decision  development  and  evaluation  to  meet  the 
challenges  of  modem  agile  warfighting.  This  paper  presents  a  Scenario-Driven  System 
Engineering  (SDSE)  approach  to  develop,  evaluate,  and  test  C2  systems.  One  key  component  is 
that  once  system  scenarios  are  specified,  the  system  can  be  simulated  without  any  programming 
and  thus  saves  significant  effort  and  time. 

SDSE  is  compatible  with  the  modem  Service-Oriented  Architecture  (SOA)  approach  to 
develop  tmstworthy  systems.  DoD  is  embracing  SOA  in  numerous  projects  such  as  the  Defense 
Information  Systems  Agency  GIG  Enterprise  Services  (GES)  with  its  component  core  services. 
The  SDSE  can  be  used  to  specify  and  analyze  system  behaviors  in  an  SOA. 

The  core  of  SDSE  is  scenario  specification  and  analyses.  A  scenario  is  specified  using  the 
ACDATE  model: 
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•  Actors  -  An  actor  is  either  an  external  user,  system  or  deviee,  or  an  internal  system, 
deviee,  eomponent  or  objeet; 

•  Conditions  -  A  eondition  is  a  predicate  used  to  trigger  an  action; 

•  Data  -  Attributes  of  aetor,  and  presenting  the  semantie  of  eondition,  event  and  aetion 

•  Aetions  -  Specified  by  the  trigger  event,  guard  condition,  the  way  to  change  the  status 
of  aetors,  and  sent  event(s)  to  some  aetors 

•  Timing  -  A  semantie  statement  about  the  relative  or  absolute  value  of  time  or  duration 

•  Events  -  External/internal  signifieant  oeeurrenees  that  may  trigger  action(s) 

Onee  system  behaviors  are  speeified,  various  statie  and  dynamie  (via  simulation)  analyses  ean 
be  performed  on  the  model; 

•  Completeness  and  eonsistency  analysis:  The  model  ean  be  used  to  identify 
ineompleteness  at  eompile  time  as  well  as  during  simulation. 

•  Performanee  evaluation:  The  model  ean  be  simulated  to  determine  system  performanee 
ineluding  throughput  and  delay. 

•  Safety  analysis:  The  model  ean  be  used  to  generate  the  event-tree  model  and  effeet- 
cause  diagram  eommonly  used  in  safety  analysis; 

•  Behavior  analysis:  The  model  ean  be  used  to  generate  the  state  model  of  the  system  and 
various  behavior  analyses  sueh  as  reachability  analysis,  which  can  be  performed  on  the 
state  model.  Eormal  verification  techniques  sueh  as  temporal  logie  ean  be  used  to 
analyze  the  state  model. 

The  SDSE  is  to  be  integrated  and  supported  by  an  automated  tool  E2E  that  is  eurrently  being 
used  in  several  experimental  projeets  by  EIS  Navy. 

2,  ACDATE  Model  -  An  Example 

This  seetion  presents  an  example  of  ACDATE  modeling  teehnique  whieh  usually  eontains  two 
steps: 

•  Deeompose  the  requirements  into  ACDATE  model  elements;  and 

•  Develop  system  seenarios  using  the  ACDATE  model  elements. 

Taking  a  battlefield  as  an  example,  eaeh  warfighting  vehicle  ean  be  treated  as  an  Actor.  Eaeh 
aetor  (warfighting  vehiele)  may  have  its  own  Data  sueh  as  “available  fuel”,  and  its  own 
Conditions  sueh  as  if  there  is  enough  fuel  to  eontinue  moving  for  20  miles.  The  given  eondition 
example  is  eonstructed  on  the  Data  ‘available  fuel’.  An  Event  “not  enough  fuef’  could  be  fired 
when  there  is  not  enough  fuel  support  subsequent  operations.  An  Action  would  be  “to  refill  the 
vehiele”. 

A  system  seenario  would  then  be  speeified  using  the  ACDATE  model  elements:  if  the  event 
“not  enough  fuel”  oeeurs,  the  aetor  “warfighting  vehiele”  shall  perform  action  “to  refill”  within 
the  time  speeified  by  the  Timing  Attribute  “within  15  minutes”. 

3,  Scenario-Based  Rapid  Simulation 

The  key  feature  of  the  rapid  simulation  is  automatie  simulation  eode  generation  onee  the 
system  scenarios  are  available.  The  simulation  eode  has  an  embedded  scheduler,  an  event  queue. 
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a  monitor,  and  a  policy  checker  to  traek  and  verify  the  alternative  system  behaviors  under 
different  environments,  as  well  as  the  impact  from  and  to  the  environment.  The  simulation  is 
discrete  event  simulation  [1][3]. 

3.1  Simulation  Engine  Architecture 

The  simulation  engine  has  two  main  parts:  system  simulator  and  the  environment  simulator. 
The  environment  simulator  simulates  the  behavior  of  the  environment.  It  can  also  simulate  the 
impaet  of  the  system  to  its  environment.  Separating  the  system  simulation  and  environment 
simulation  has  several  significant  advantages:  It  offers  the  opportunity  to  observe  the  behavior  of 
the  system  under  testing  (SUT)  under  different  loads  by  varying  the  environment  simulator. 
Speeifically,  robustness,  reliability  and  scalability  of  the  system  can  be  determined  by  generating 
various  inputs  to  drive  the  system,  e.g.,  generating  an  incorreet  input  can  evaluate  the  system’s 
robustness,  and  generating  the  input  according  to  the  operational  profile  will  determine  the 
system’s  reliability,  and  generating  inputs  of  various  sizes  to  determine  the  system  scalability. 


The  simulation  engine  arehiteeture  is  illustrated  by  Figure  1.  The  ACDATE  model  elements 
form  an  entity  pool.  The  execution  of  each  scenario,  whieh  is  seheduled  by  the  scheduler,  will 
aceess  the  entity  pool  to  read  or  update  their  internal  status.  Events  may  be  emitted  during  the 
execution  of  a  scenario,  which  may  in  turn  invoke  the  execution  of  other  seenarios.  An  event 
queue  is  maintained  to  process  all  the  events  emitted  by  scenarios  or  the  environment.  The 
seheduler  will  drive  the  monitor  or  the  policy  checker  properly  to  track  all  the  activities  or  do 
runtime  policy  verification  respectively. 


Figure  1:  Scenario  Based  Simuiation  Architecture 


The  core  of  the  seheduler  is  a  virtual  maehine  (VM)  that  enables  fine-grained  debugging 
capability.  The  execution  of  any  scenario  can  be  slow-played  and  the  internal  status  of  eaeh 
entity  can  be  set-up  to  any  desired  value  to  offer  the  opportunity  to  manipulate  the  simulation 
and  observe  rare  oceurrenee  or  exceptional  behaviors. 

The  monitor  will  track  all  the  activities  and  the  information  will  be  used  to  generate  the  event- 
tree,  the  state  diagram  for  each  actor  or  the  entire  system.  Eor  any  potential  or  real  malfunction, 
the  monitor  or  the  policy  checker  will  report  warnings  to  indicate  either  there  is  a  completeness 
and  consistency  breach  or  concurrency  or  security  policy  violation.  The  monitor  will  also  record 


3 


the  time  elapsed  that  ean  be  used  to  form  the  performanee  evaluation  of  eaeh  actor  or  the  entire 
system. 

The  simulation  works  as  follows.  An  event  residing  in  the  event  queue,  which  might  be 
emitted  by  the  environment  or  the  system,  is  picked  up  by  the  scheduler  and  processed.  The 
event  may  trigger  the  execution  of  a  scenario,  which  will  be  either  concurrent  with  the  execution 
of  other  scenarios,  or  occupy  the  VM  before  it  finishes,  due  to  the  different  scheduling  policy. 
The  execution  course  of  the  scenario  may  be  determined  by  the  internal  status  of  some  entities. 
The  scheduler  will  read  or  update  the  corresponding  internal  status  upon  the  request  of  the 
scenario.  New  events  will  be  emitted  on  behalf  of  the  scenario  during  execution,  which  will  be 
appended  to  the  end  of  the  event  queue.  The  new  events  might  be  communication  among 
components  inside  the  system  or  an  outgoing  event  to  the  environment.  Details  of  the  execution, 
such  as  time  information,  action  sequence,  and  event  sequences  will  be  recorded  through  the 
monitor,  which  is  available  to  postmortem  analyses.  If  a  policy  is  registered  into  the  scheduler, 
corresponding  policy  checking  will  be  invoked  by  the  policy  checker  at  runtime. 

3,2  Simulation  Code  Generation 

The  simulation  code  is  generated  based  on  the  scenario  specification,  which  includes  the 
ACDATE  definition  and  scenario  description.  Each  ACDATE  model  element  will  be  translated 
to  an  object  with  the  attributes  defined  in  the  specification.  Instrumentation  code  will  be  inserted 
to  the  objects  to  interface  with  the  monitor  and  policy  checker.  Each  scenario  will  be  translated 
to  a  procedure  that  is  basically  a  sequence  of  operations  on  the  ACDATE  objects  or  emitting 
events.  Similarly,  instrumentation  code  will  be  inserted  to  the  procedure  to  interface  with  the 
scheduler,  for  scheduling  the  concurrent  execution,  and  event  queue  for  emitting  new  events. 
Table  1  shows  a  sample  simulation  code  that  is  automatically  generated  with  instrumentation 
code  that  interfaces  with  the  scheduler,  event  queue,  monitor,  or  policy  checker. 


scenario_5  =  function (co_routine_name,  platform)  //  a  scenario 
coroutine .  yield  0  ;  //  interface  to  scheduler 


event_4 : AddDestination ( 1 )  ; 
event_4 : emit ( )  ; 


/  /  interface  to  monitor 
/  /  interface  to  event  queue 


data_3 .  value  =  10  00;  /  /  data_3 :  set  ( )  will  be  invoked  and 

/  /  interface  to  monitor  embedded  there 

act  ion_l0  :  bef  ore_do  ( co_rout  ine_name,  platform);  //  interface  to  policy  checker  embedded  here 
action_10_dummy_func (co_routine_name,  platform) ; 
action_10 : after_do (co_routine_name,  platform) ; 

timer  [platform]  =  timer  [platform]  +  unit;  //  advance  and  record  system  time 


Table  1  Sample  Simulation  Code 


4,  Simulation  for  Dynamic  Analyses 

Various  dynamic  analyses  are  enabled  by  simulation,  e.g.,  completeness  and  consistency 
(C&C)  analysis,  performance  analysis,  safety  analysis,  and  behavior  analysis. 

The  dynamic  C&C  analysis  complement  static  C&C  checking  [6]  because  some 
incompleteness  or  inconsistency  can  only  be  observed  during  runtime  when  concurrency  comes 
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into  play.  The  simulation  will  record  all  the  potential  inconsistency  and  incompleteness  such  as 
the  simulation  encounters  a  situation  where  there  is  no  related  instruction  in  the  specification;  or 
an  action  did  not  change  the  state  of  any  actors,  which  may  imply  that  the  system  is  at  an 
abnormal  state  that  responses  to  no  input.  In  a  recent  experiment  with  a  C2  system  that  has 
around  1000  entities  and  120  scenarios,  it  takes  3  minutes  to  generate  and  execute  the  simulation 
and  it  detected  around  200  bugs  related  to  incompleteness  or  inconsistency. 

During  the  simulation,  system  time  will  be  recorded  for  each  action  when  it  starts  or  ends, 
event  when  it  is  emitted  or  handled,  and  data  when  the  value  changes.  The  recorded  time 
information  can  be  used  for  performance  analysis  such  as  the  throughput  and  average  delay  of 
the  system. 

Event  sequence  can  be  useful  for  safety  analysis.  Figure  2  shows  a  sample  event  sequence  tree 
which  is  automatically  generated  by  simulation.  The  effect-cause  diagram  [7]  can  be  also 
automatically  generated  by  examining  all  the  event  trees  related  to  a  system  to  link  from  an 
effect  to  its  possible  causes.  Effect-cause  diagram  is  similar  to  fault  tree  [5]  except  that  the 
elements  on  the  diagram  are  analytic  entries.  By  combining  event  sequence  tree  and  effect-cause 
diagram,  one  can  pinpoint  which  system  components  that  failed  during  failure  analysis. 


Figure  2  Sample  Event  Sequence  Tree 
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Behavior  analysis,  such  as  reachability  [8]  analysis,  is  usually  performed  on  state  model. 
There  is  a  natural  mapping  from  the  state  model  generated  from  the  simulation  (SMM)  [7]  to 
UML’s  Statechart  [2],  which  can  then  be  fed  into  various  UML  tools  for  further  analyses.  It  is 
also  possible  to  perform  Linear  Temporal  Logic  (LTL)  analysis  and  model  checking  using  SPIN 
[7]  on  the  state  model  to  detect  deadlock  or  other  malfunctions. 

5,  Conclusion 

This  paper  proposes  a  systematic  process  to  perform  variety  kinds  of  dynamic  analyses  based 
on  scenario  specification.  Once  system  scenarios  are  specified,  the  simulation  code  can  be 
automatically  generated,  and  the  system  can  be  simulated  without  any  additional  programming. 
The  simulation  can  be  used  to  perform  various  dynamic  analyses  including  C&C  checking, 
safety  analysis,  and  performance  analysis.  The  SDSE  is  being  integrated  into  an  automated  tool 
E2E. 
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Real-Time  Distributed  Network- 

Centric  Warfare 

•  System  modeling  and  simulation  from  the  system 
requirement  important  for  network-centric  warfare 

•  System  modeling  and  simulation  are  usually  expensive 
for  real-time  distributed  network-centric  warfare 

•  Many  specification  and  simulation  packages  are 
available  such  as  SDL  that  can  model  and  simulate  the 
target  system  from  requirements.  But  they  are  often 
design-oriented. 

•  Our  scenario-driven  system  engineering  approach 
provides  a  way  to  perform  system  modeling  and 
simulation  rapidly  based  on  system  requirements. 
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Comparisons  Between  SDL  and 
Scenario  ACDATE  Model 
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SDSE  and  Command  &  Control 

Systems 

•  Future  Command  and  Control  (C2)  systems 
need  to  operate  within  an  integrated  grid-based 
network-centric  environment  (GIG)  that  allows 
rapid  decision  development  and  evaluation  to 
meet  the  challenges  of  modern  agile  warfighting. 

•  A  Scenario-Driven  System  Engineering  (SDSE) 
approach  is  proposed  to  develop,  evaluate,  and 
test  C2  systems 

•  Once  system  scenarios  are  specified,  the 
system  can  be  simulated  without  any 
programming  and  thus  saves  significant  effort 
and  time. 
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SDSE  Features 


•  Compatible  with  the  Service-Oriented 
Architecture  (SOA)  to  develop  trustworthy 
systems 

•  Can  be  used  to  specify  and  analyze 
system  behaviors  in  an  SOA. 

•  Core:  scenario  specification  and  analyses 
based  on  ACDATE  model. 
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ACDATE  Model 

•  Actors  -  An  actor  is  either  an  external  user,  system  or 
device,  or  an  internal  system,  device,  component  or 
object; 

•  Conditions  -  A  condition  is  a  predicate  used  to  trigger  an 
action; 

•  Data  -  Attributes  of  actor,  and  presenting  the  semantic 
of  condition,  event  and  action 

•  Actions  -  Specified  by  the  trigger  event,  guard  condition, 
the  way  to  change  the  status  of  actors,  and  sent  event(s) 
to  some  actors 

•  Timing  -  A  semantic  statement  about  the  relative  or 
absolute  value  of  time  or  duration 

•  Events  -  External/internal  significant  occurrences  that 
may  trigger  action(s) 

2004-5-22 
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A  Sample  Scenario 


using 


using 

using 

using 

if( 

then 

-[ 

do 

do 

} 

else 


do 


1 


DriverDoor.DriverDoorlsLockecj  | 


PassengerDccr  PassengerDoorlsLocked ) 


AlarmJurnOnAlarnn| 

Horn ,  MakeHornBeepOnce  ( 


Horn.HornStatus) 


Horn,MakeHornBeepThreeTiines(  Horn.Horr  Status) 


A  scenario  “when  driver  door  is  locked  and  passenger  door  is  locked,  if  remote 
controlled  is  pressed  unlock,  then  the  driver  door  is  open”  can  be  specified  using 
Scenario  Specification  Language  as  above 


2004-5-22 


7 


Analyses  based  on 
ACDATE/Scenario  Model 

•  Based  on  the  ACDATE/Scenario  model,  a 
variety  of  static  and  dynamic  (via  simulation) 
analyses  can  be  performed: 

-  Completeness  and  consistency  analysis 

-  Performance  evaluation 

-  Safety  analysis 

-  Behavior  analysis 

-  Policy  specification  and  enforcement 

•  The  SDSE  is  to  be  integrated  and  supported  by 
an  automated  tool  E2E  that  is  currently  being 
used  in  several  experimental  projects  by  US 
Navy 
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Scenario  Tool  Input  Interface 
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static  C&C  Analysis 

•  Once  the  system  ACDATE/Scenario 
model  is  ready,  one  can  easily  using  our 
automated  tool  to  perform  completeness 
and  consistency  analysis  to  see  if  there  is 
any  problem  in  system  modeling 

•  Static  C&C  analysis  can  discover  a  large 
amount  of  incompleteness  and 
inconsistency  problems  that  are  hard  for 
engineers  to  detect 
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static  C&C  Analysis  Tool 
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Experiment  Results  of  Static  C&C 

Results 


Syitemv 

Type 

ExpenmeiitaL'' 

Industrial 

^of 

Scenarios 

^of 

Conch. 

F  of  Missmg 

C-E 

Combination 

^of 

Co’t'ering 

Scenarios 

WTiole/ 

Partial 

System 

CarAlaim 

S^'steiiL^ 

Ceutialized 

application. 

Experimental 

13 

12 

547 

40 

mole 

Bankii’.^  S^v'^tem 

Di^^tfibuted 

application 

23 

2 

0 

0 

Whole 

ICS 

Di^^ti'ibuted 

real-time 

application. 

Industrial 

140 

15 

396SOO 

29 

Uliole 

RCS 

Di^^tiibuted 

real-time 

application. 

54 

10 

1792 

4 

Partial 
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Di^^tiibuted 
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application. 

10 

19 

m60Z0 

S 

mole 

Communicatiou 

Prccesscf 

Embedded 

distributed 

application 

31 

33 

1.29  MO-^ 

27 

Partial 

Scenario-Based  Simulation 

Architecture 


•  Scenario-based  simulation  is  divided  into 
two  major  parts 

-  Environment  Simulator 

•  The  behavior  of  the  environment 


Rationale  for  Separating  Environment 

And  System  Simulation 

•  It  offers  the  opportunity  to  observe  the  behavior  of  the 
system  under  testing  (SUT)  under  different  loads  by 
varying  the  environment  simulator 

•  Robustness,  reliability  and  scalability  of  the  system  can 
be  determined  by  generating  various  inputs  to  drive  the 
system 

-  Generating  an  incorrect  input  can  evaluate  the  system’s 
robustness 

-  Generating  the  input  according  to  the  operationai  profile  will 
determine  the  system’s  reliability 

-  Generating  inputs  of  various  sizes  to  determine  the  system 
scalability 
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Simulation  Engine  Architecture 
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Simulation  Code  Generation 


•  The  simulation  code  is  generated  based  on  the  scenario 
specification,  which  includes  the  ACDATE  definition  and 
scenario  description 

•  Each  ACDATE  model  element  will  be  translated  to  an 
object  with  the  attributes  defined  in  the  specification 

•  Instrumentation  code  will  be  inserted  to  the  objects  to 
interface  with  the  monitor  and  policy  checker 

•  Each  scenario  will  be  translated  to  a  procedure  that  is 
basically  a  sequence  of  operations  on  the  ACDATE 
objects  or  emitting  events. 

•  With  these  automated  simulation  code  generation, 
previously  Excel-spreadsheet  based  real-time  distributed 
network-centric  C2  systems  can  be  simulated  without 
any  additional  programming  effort  saving  significant 
effort. 
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Simulation  Code  Generation  - 

Sample  Scenario 


using 

using 

using 

using 

using 


if( 

then 

-[ 


do 

do 


} 

gise 

■[ 


do 


1 


DriverDoor.DriverDoorlsLockecj  | 


PassengerDcMr.PassengerDoorlsLocked ) 


AlarniJurnCnAlarnfil 
Horn ,  MakeHornBeepOnce  ( 


Horn.HornStatus) 


Hor  n ,  MakoHor  nBoepThresT  i  ines(  Horn ,  Horn  5  tatus) 


A  scenario  “when  driver  door  is  locked  and  passenger  door  is  locked,  if  remote 
coliW^fd^ is  pressed  unlock,  then  the  driver  door  is  open”  can  be  specified  usingi 
Scenario  Specification  Language  as  above 


Simulation  Code  Generation 
Example  -  Generated  Code 

scenario_5  =  function(co_routine_name,  platform)  //  a  scenario 
coroutine.yieldO;  //  interface  to  scheduier ... 

if  (condition_1 1.Eval()  ||  condition_17.Eval() )  //condition  evaluation 
then 

action_10:before_do(co_routine_name,  platform);  //interface  to  poiicy 

//checker  embedded  here 

action_10_dummy_func(co_routine_name,  platform);  //  turn  on  alarm 
action_1 0:after_do(co_routine_name,  platform); 

timer[platform]  =  timer[platform]  +  unit;  //  advance  and  record  system 

//time ... 

action_17:  action_10:before_do(co_routine_name,  platform); 
action_17_dummy_func(co_routine_name,  platform);  // beep  once 
action_17:after_do(co_routine_name,  platform); 
timer[platform]  =  timer[platform]  +  unit; 
else 

action_20:  action_10:before_do(co_routine_name,  platform); 
action_20_dummy_func(co_routine_name,  platform);  //  beep  three  times 
action_20:after_do(co_routine_name,  platform); 
timer[platform]  =  timer[platform]  +  unit; 
end 
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Sample  Simulation  Result 


Execution  Sequence 


[30]  Event  E1_ReceiveFireOrder/is  Generated. 

[31]  Event  E13_ReceiveFireOrqer  with  Event  Instance  ID  of  12 
Arriving  at  30 

[31]  Begin  Doing  Action  Action  13_ReceiveFireOrder 

[32]  Condition  Vig3_ChancellorsvilleCanShoot  is  Evaluated  to  Be  False 


[33]  Begin  Doing  Action  Action3_MakeDecisioi 

[34]  Condition  Con\liton27_DecideRejectMissio>hi^  Evaluated  to  Be 

True 
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Dynamic  Analyses  Performed 
Based  on  Simulation 

•  Once  simulation  is  performed,  variety 
kinds  of  analyses  can  be  carried  out  based 
on  simulation,  both  runtime  and  off-line: 

-  Policy  specification  and  enforcement 

-  Dynamic  C&C  analysis 

-  Performance  analysis 

-  Safety  analysis 

-  Behavior  analysis 
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Policy  Specification  and 
Enforcement 

•  One  can  specify  kinds  of  policies  in  the 
system  ACDATE/Scenario  model  using 
our  automated  scenario  tool 

•  Once  policies  are  specified,  simulation  can 
dynamically  check  and  enforce  the 
specified  policies  at  runtime. 

•  Any  policy  violation  will  be  reported  and 
recorded  in  a  log  file. 
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Policy  Specification 


•  Policy  2:  Supporting  Arms  Coordinator  (SAC)  must  NOT  issue  a  Fire 
Order  if  SOF  Team  has  not  laid  down 

•  Specification 


jPV  CBL  Policy  Specification 


Policy 


Policy  Name 


Policy02 


Policy  Type  |  Must  Not  Do 


Policy  Actor  I  E5SE_SAC 


Policy  Action  |  Vig3_A_lssueFireOrder 
Policy  Data 


Policy  Condition  |  Vig3_50FTeamlsNotLainDown 
Compensation  [PoNothing 


Policy  Hierarchy 


[  Analyze  ~] 

[  Update  I 


Cancel 


Enforce  Policy 


Delete 


ID 

Type 

Actor 

Data/Status 

Action 

Condition 

858 

Must  Not  . . 

Briaade  TOC  FSO 

Vig3  A  IssueCFFCommand 

Vig3  T argetlsNotFound 

842 

Must  Do 

System 

Sequence 

TRUE 

. . . 

865 

Must  Not  . . 

USS  JOHN  S  MCCAIN 

Vig3  A  McClainIssueFireGu. . . 

Vlg3  CTFIsNotApproved 

850 

Must  Not  . . 

ESSE  SAC 

Vlg3  A  IssueFireOrder 

Vig3  SOFTeamIsWithInFie 

851 

Must  Do 

System 

Sequence 

TRUE 

854 

Must  Not  . . 

USS  CHANCELLORS  VILLE 

Via3  A  VilleRelectMission 

Vig3  ChancellorsvilleCanS 

860 

Must  Not  . . 

ESSE  SAC 

Vlg3  A  IssueFireOrder 

Vig3  CFFIsNotReceived 

861 

Must  Not  . . 

SOF  Observer 

Viq3  A  ObserveTarqetStatus 

Vlg3  MFRIsNotReceived 

<  > 
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Acceptable  Scenario 


Items  List  Vig3_SC09_ReceiveCFF 
ToolBox  @  |X| 


Keywords 


Symbols 


ACDATEs 


Misc 


Old  Style 


in..  Ac  I  !': 

vr  >  [_■  VC 

exe.V., 
&&  And 


0 


A.c'o  _:in  c: 


□ 


using  ACTOR  :ESSE_SAC 
using  ACTOR  :SOF_Team 

do  ACTION  :SOF_Team,Vig3_A_Retreat 
do  ACTION :  SOF_T ea  m .  V  ig3_A_L  leDown 

do  ACTION :  ESSE_SAC .  V  ig3_A_Deter  m  IneBestWeapon 
do  ACTION  :ESSE_SAC. Vig3_A_CheckSOFTeamOutOfTan 
do  ACTION  :ESSE_SAC.Vig3_A_CheckSOFTeamLiedov^ 

emit  EVENT :ESSE_SAC.Vig3_E10_IssueFireOrder 


SOF  Team  lies  down 
before  Fire  Order 
is  issued 


No  policy  violation 
detected 
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W  WarningLog  - 

Notepad 

File  Edit 

Format 

View  Help 

1 

3 

1 

15 

Vi g3_A_inputTargetlnf o  changecT/rto  data  value 

1 

5 

1 

15 

Vi g3_A_sendTargetinf 0  changejd^o  data  value 

1 

8 

1 

15 

vi g3_A_ReadTargetinf 0  chari^^  no  data  value 

1 

11 

1 

15 

Vi g3 WissueCFFCommand  cja^ged  no  data  value 

1 

17 

1 

15 

vi g3_A_Determi neeestwe^fion  changed  no  data  value 

1 

18 

1 

15 

Vi g3_A_CheckSOFTeamOjdifofTargetArea  changed  no  data  value 

1 

19 

1 

15 

Vig3_A^CheckSOFTe^|pLiedown  changed  no  data  value 

1 

21 

1 

15 

vi g3_A_issueFi reorder  changed  no  data  value 

1 

25 

1 

15 

Vi  g3^cc_MakeDeci  si  on  changed  no  data  value 

1 

27 

1 

15 

vig3wvilleRejectMission  changed  no  data  value 

1 

33 

1 

15 

vi g3_A_Mccl ai nissueFi recunorder  changed  no  data  value 

1 

44 

1 

15 

Vi g3 WObserveTargetStatus  changed  no  data  value 

1 

46 

1 

15 

Vi g3_A_inputTargetstatus  changed  no  data  value 

1 

52 

1 

15 

vi g3_A_ReadReportDestroyed  changed  no  data  value 

1  Ln  11,  Coll  1 

Scenario  Violating  Policy 


Items  List  Vig3_SC09 
ToolBox  @  lx] 


Keywords 


Symbols 


ACDATEs 


Misc 


old  Style 


n 


d  -  -Q  - 

V  - 

Jo '  i  I 

-hi 

&&  r 


i  L-c  npl'-h 


_ReceiveCFF 

using  ACTOR  :ESSE_SAC 
using  ACTOR  :SOF_Team 

do  ACTION ;SOF_Team.Vig3_A_Retreat 

do  ACTION :  ESSE_SAC .  V  ig3_A_Deter  m  ineBestWeapon 
do  ACTION  :ESSE_SAC. Vig3_A_CheckSOFTeamOutOfTan 
do  ACTION  :ESSE_SAC. Vig3_A_CheckSOFTeamLiedo 

emit  EVENT ;ESSE_SAC.Vig3_E10_IssueFireOrder 


SOF  Team  doesn’t 
lie  down  before  Fire 
Order  is  issued 


Policy  violation 
detected 


P  WarningLog  -  Notepad 

1  Rie  Edit  Format  View  Help 

7 

< 
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1 

15 

5 

1 

15 

8 

1 

15 

11 

1 

15 

16 

1 

15 

17 

1 

15 

18 

1 

15 

20 

3 

15 

20 

1 

15 

24 

1 

15 

26 

1 

15 

32 

1 

15 

43 

1 

15 

45 

1 

15 

51 

1 

15 

vig3^ft_lnputTargetlnfo  chan^d  no  data  value 
Vi g3_A_SendTargetlnf o  chanoed  no  data  value 
vig3_A_ReadTargetinfo  changed  no  data  value 
Vi g3_A_lssueCFFCommand  Ranged  no  data  value 
Vi g3_A_Determi neBestWe^on  changed  no  data  value 
vig3_A_checksoFTeamou^fofTargetArea  changed  no  data  value 
Vi g3^A_checkSOFTeaniL/i edown  changed  no  data  value 
Policy  847  Failed..^. 

Vi g3_A_issueFi reorder  changed  no  data  value 
Vi g3_A_MakeDeci si  on  changed  no  data  value 
vig3_A_villeRejectMission  changed  no  data  value 
vig3_A_McclainlssueFireGunorder  changed  no  data  value 
Vi g3_A_ObserveTargetStatus  changed  no  data  value 
vig3_A_inputTargetstatus  changed  no  data  value 
vig3_A_ReadReportDestroyed  changed  no  data  value 


Ln  16,  Col  1 


Dynamic  C&C  analysis 


•  Some  incompleteness  or  inconsistency  can  only 
be  observed  during  runtime  when  concurrency 
comes  into  play 

•  In  a  recent  experiment  with  a  real  time  network¬ 
centric  C2  system  that  has  around  1000  entities 
and  120  scenarios,  it  takes  3  minutes  to 
generate  and  execute  the  simulation  and  it 
detected  around  200  bugs  related  to 
incompleteness  or  inconsistency. 
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Performance  Analysis 


•  During  the  simulation,  system  time  will  be 
recorded  for  each  action  when  it  starts  or 
ends,  event  when  it  is  emitted  or  handled, 
and  data  when  the  value  changes. 

•  The  recorded  time  information  can  be 
used  for  performance  analysis  such  as  the 
throughput  and  delay  of  the  system 
processes. 
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Safety  Analysis 

•  Event  sequence  can  be  useful  for  safety 
analysis. 

•  By  using  event  sequence  tree  and 
traditional  event  tree,  one  can  pinpoint 
which  system  components  that  failed 
during  failure  analysis. 
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Sample  Event  Sequence  Tree 
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Behavior  analysis 


•  Reachability  analysis 

•  State  model  generation 

•  Linear  Temporal  Logic  (LTL)  analysis 

•  Model  checking  using  SPIN 

•  Sequence  diagram  generation 
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state  Model  Generation 


•  We  can  generate  the  state  model  from  the 
result  of  simulation 

•  Information  contained  in  one  entry  of  the 
simulation  result 

-Time  stamp 

-  Starting  and  ending  system  state 

-  Action  performed 

-  Event  that  triggers  the  action 
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state  Model  Generation 

•  From  information  provided  in  the  result  of  simulation,  we 
can  get  the  global  state  transitions  and  single  actor  state 
transitions 

•  But  there  is  something  more  for  the  single  actor  state 
transition  generation  -  guard  condition,  i.e.  some  state 
transition  can  happen  when  some  other  actors  are  in 
certain  conditions.  For  example,  alarm  can  not  be  turned 
on  when  the  driver’s  door  is  open. 

•  State  transition  for  global  system: 

-  (starting  global  state)  -  external  event/triggered  actions 
(ending  global  state) 

•  State  transition  for  single  actor 

-  (starting  actor  state)  -  external  event[guard  condition]/triggered 
actions  (ending  actor  state) 
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Model  Checking  with  SPIN 

•  One  can  generate  the  single  actor  state 
models  automatically  from  simulation 

•  One  can  also  generate  the  single  actor 
state  models  from  requirements  or  design 
manually 

•  Two  sets  of  state  models  can  put  into 
SPIN  to  perform  cross  checking 
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Sample  State  Model 


A  B 

C 

D 

E  F  G 

1  iStartState  lEvent 

GuardCondition 

Actions 

EndState 

2  Door  Closed  Not  Loc  eLock/ArmButtonPressedWRerr 

Door  Closed  Not  Locked 

(Lock  driver  door)  (Lock  pc  Door  Closed  And  Locked 

3  Door  Closed  And  Loc  eLockPassengerDoorWKey 

No  Guard 

No  Action 

Door  Closed  And  Locked 

Door  Closed  And  Loc  eLock/ArmButtonPressedWRen 

Door  Closed  And  Lockec 

(Beep  Three  Times) 

Door  Closed  And  Locked 

5  Door  Closed  And  Loc  eLock/ArmButtonPressedWRen 

Door  Closed  And  Lockec 

(Beep  Three  Times) 

Door  Closed  And  Locked 

6  Door  Closed  And  Loc  eLock/ArmButtonPressedWRerr  Door  Closed  And  Lockec  (Beep  Three  Times) 

Door  Closed  And  Locked 

7  Door  Closed  And  Loc  eLock/ArmButtonPressedWRen 

Door  Closed  And  Lockec 

(Beep  Three  Times) 

Door  Closed  And  Locked 

8  Door  Closed  And  Loc  eLock/ArmButtonPressedWRen 

Door  Closed  And  Lockec 

(Beep  Three  Times) 

Door  Closed  And  Locked 

This  is  a  sample  state  model  for  the  Driver’s  Door 
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Generated  Sequence  Diagram 

Sample 


CarKev 

E9  Handler  Action 

eLockPassengerDoorWithCarKey 

i - N 

I  I 
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Conclusion 


•  A  systematic  process  to  perform  variety  kinds  of 
static  and  dynamic  analyses  based  on  scenario 
specification 

•  Once  system  scenarios  are  specified,  the 
simulation  code  can  be  automatically  generated, 
and  the  system  can  be  simulated  without  any 
additional  programming 


•  The  simulation  can  be  used  to  perform  various 
dynamic  analyses  including  C&C  checking, 
safety  analysis,  and  performance  analysis.  The 


SDSE  is  being  integrated  into  an  automated  tooi 
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